RHIDP-14000: add integration tests for /v1/responses endpoint by Jdubrick · Pull Request #1933 · lightspeed-core/lightspeed-stack

Jdubrick · 2026-06-15T18:11:05Z

Description

Added 4 integration tests for the /v1/responses endpoint that exercise the handler → DB persistence path with real configuration and an in-memory SQLite database (Llama Stack client remains mocked)
Non-streaming success: verifies UserConversation and UserTurn rows are correctly created with proper user ID, model, provider, message count, and response ID
Shield-blocked moderation: verifies the modr_ prefix branching logic sets last_response_id = None on the conversation while still recording the moderation turn for auditing, and confirms the LLM is never invoked
store=False: verifies zero DB records are written when persistence is explicitly disabled
Streaming shield-blocked: verifies SSE event format (response.created, response.completed, [DONE]) and that post-stream DB persistence correctly writes the moderation turn after the stream is fully consumed

Type of change

Tools used to create PR

Identify any AI code assistants used in this PR (for transparency and review context)

Assisted-by: Cursor
Generated by: (e.g., tool name and version; N/A if not used)

Related Tickets & Documents

Related Issue # https://redhat.atlassian.net/browse/RHIDP-14000
Closes # https://redhat.atlassian.net/browse/RHIDP-14000

Checklist before requesting a review

I have performed a self-review of my code.
PR has passed all pre-merge test jobs.
If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

Summary by CodeRabbit

Release Notes

Tests
- Added integration tests for the responses endpoint covering end-to-end scenarios: successful response creation with conversation/turn persistence, shield-blocked responses with moderation handling and safety text, streaming response delivery with SSE events, and store=False configuration bypassing database operations, ensuring reliability and correct data handling across all response types.

Co-authored-by: Cursor <cursoragent@cursor.com>

coderabbitai · 2026-06-15T18:11:20Z

Warning

Review limit reached

@Jdubrick, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 6 minutes and 41 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 64d1a692-08c6-4e8e-94bc-0735f2eee8c9

📥 Commits

Reviewing files that changed from the base of the PR and between aafdb2c and fda1bbe.

📒 Files selected for processing (1)

tests/integration/endpoints/test_responses_integration.py

Walkthrough

Adds a new integration test module tests/integration/endpoints/test_responses_integration.py with four async tests exercising the /v1/responses endpoint handler-to-DB path. Uses an in-memory SQLite database and a fully mocked Llama Stack client to cover non-streaming success, shield-blocked non-streaming, store=False, and streaming shield-blocked scenarios.

Changes

Integration tests for /v1/responses endpoint

Layer / File(s)	Summary
Test fixtures, mock client, and patching helpers `tests/integration/endpoints/test_responses_integration.py`	Defines `MOCK_AUTH`, `MOCK_CONV_ID`, `NORMALIZED_CONV_ID`, `_RESPONSE_DUMP` constants; `_build_mock_client` constructs a mocked `AsyncLlamaStackClient` with stubs for `responses.create`, `models.list`, `shields.list`, `vector_stores.list`, and `conversations.create`; `_patch_client_holders` replaces `AsyncLlamaStackClientHolder` across endpoint modules and bypasses `ResponsesContext` Pydantic validation; `_setup_test` and `_configure_shield_blocked` wire per-test isolation and shield simulation.
Non-streaming path tests: success, shield-blocked, store=False `tests/integration/endpoints/test_responses_integration.py`	`test_non_streaming_success_persists_conversation_and_turn` asserts `ResponsesResponse` fields and verifies `UserConversation`/`UserTurn` DB rows with provider/model normalization and counters. `test_shield_blocked_persists_moderation_turn` asserts moderation-derived response id, safety text, `responses.create` not called, and `last_response_id=None`. `test_store_false_skips_db_persistence` confirms no DB rows are created when `store=False`.
Streaming shield-blocked SSE test `tests/integration/endpoints/test_responses_integration.py`	`test_streaming_blocked_returns_sse_and_persists_turn` reads the `StreamingResponse` body and asserts `response.created`, `response.completed`, `[DONE]` SSE events and safety text, confirms `responses.create` is not called, and verifies a single `UserTurn` row with the moderation id and `last_response_id=None`.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding integration tests for the /v1/responses endpoint, which aligns with the changeset.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

✨ Simplify code

Create PR with simplified code

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/endpoints/test_responses_integration.py`:
- Around line 351-354: The current assertions on lines 351-354 only verify that
individual substrings are present in body_str but do not enforce the order in
which the SSE events appear. Replace the individual substring assertions with a
test that explicitly validates the event sequence order. Verify that "event:
response.created" appears before "event: response.completed", which in turn
appears before "data: [DONE]", and that "Content blocked by safety shield"
appears in the appropriate position within this sequence. This ensures the
streaming contract properly validates that events arrive in the expected order,
not just that they exist somewhere in the response.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 2da72afa-7c1a-43af-b69d-c7ad718e1cc7

📥 Commits

Reviewing files that changed from the base of the PR and between f1ccb35 and aafdb2c.

📒 Files selected for processing (1)

tests/integration/endpoints/test_responses_integration.py

📜 Review details

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)

GitHub Check: Pylinter
GitHub Check: build-pr
GitHub Check: unit_tests (3.12)
GitHub Check: unit_tests (3.13)
GitHub Check: E2E: library mode / ci / group 1
GitHub Check: E2E: library mode / ci / group 2
GitHub Check: E2E: library mode / ci / group 3
GitHub Check: E2E: server mode / ci / group 3
GitHub Check: E2E: server mode / ci / group 2
GitHub Check: E2E Tests for Lightspeed Evaluation job
GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-on-pull-request
GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-0-6-on-pull-request

🧰 Additional context used

📓 Path-based instructions (1)

tests/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

tests/**/*.py: Use pytest for all unit and integration tests; do not use unittest
Use pytest.mark.asyncio marker for async tests

Files:

tests/integration/endpoints/test_responses_integration.py

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

tisnik

LGTM

RHIDP-14000: add integration tests for /v1/responses endpoint

aafdb2c

Co-authored-by: Cursor <cursoragent@cursor.com>

coderabbitai Bot reviewed Jun 15, 2026

View reviewed changes

Comment thread tests/integration/endpoints/test_responses_integration.py Outdated

check for ordering in streaming result

fda1bbe

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

tisnik approved these changes Jun 16, 2026

View reviewed changes

tisnik merged commit d8d704d into lightspeed-core:main Jun 16, 2026
32 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

RHIDP-14000: add integration tests for /v1/responses endpoint#1933

RHIDP-14000: add integration tests for /v1/responses endpoint#1933
tisnik merged 2 commits into
lightspeed-core:mainfrom
Jdubrick:RHIDP-14000-responses-integration-tests

Jdubrick commented Jun 15, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jun 15, 2026 •

edited

Loading

Review limit reached

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

tisnik left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Jdubrick commented Jun 15, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Tools used to create PR

Related Tickets & Documents

Checklist before requesting a review

Testing

Summary by CodeRabbit

Release Notes

Uh oh!

coderabbitai Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Walkthrough

Changes

Estimated code review effort

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tisnik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Jdubrick commented Jun 15, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 15, 2026 •

edited

Loading